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(54) Picture coding and Interpolation apparatus 

f57^ A Dicture coding apparatus operates on blocks of pixels with an orthogonal matrix transformation 4, to generate data a 
StoS^ Sn ent oHhe Wock and subsequently reduces the redundancy of the data by quantisation 6, and entropy 
JiS^n^ encoding. Addrtional manipulation of the picture signal such as ^^^^^ 

Sn?f Blaring etc is achieved by multiplying the orthogonal transform matrix wrth another matrix wrthm coefficients 
memory 5, prior to operation on the signal. 

Motion compensation may be employed (Fig 3). (*n nicrrptP 

Also described (Fig 5) is an interpolation arrangement wherein the orthogonal matrix transformation (e.g. Discrete 
Cosine followed by an interpolation matrix transformation so as to change the number of pixeis. These 
two matrices may be combined so that a single transformation is applied to the signal. 





1 

i 




2 

i 




3 

i 




TV 

CAMERA 




Y/C 

SEPARATION 
CKT 




FRAME 
MEMORY 



15 



13 







I 


i 








Y/C 




FRAME 








ENC00ER 




MEMORY 



FIG. 1 



COEFFICIENTS 
MEMORY 



ORTHOGONAL 
TRANSFORM 
CKT 



QUANTIZER 



ENTROPY 
CODER 



11 



10 



INVERSE 
ORTHOGONAL 
TRANSFORM 
CKT 



INVERSE 




ENTROPY 


QUANTIZER 




0EC0DER 



COEFFICIENTS 
MEMORY 



—12 



111 



CONTROLLER 



_8 



VIDEO 
FILE. 

/VIDEO \ 

Vubrary; 



17 



KEYB0AR0\ 



o 

CD 

ro 



CD 
CD 



2211691 

1/5 



in 



Id 



COEFFICIENTS 
MEMORY 




ORTHOGONAL 

TRANSFORM 

CKT 







ONAL 
ORM 




NTS 




LU 


INVERSE 
ORTHOQ 
TRANSF 
CKT 
















2211691 

2/5 



JQ 
m 



a 



ooooo 
o o o oo- 
oo 

06 
0 o 
00 
00 



-00 
-000 



00 
00 
00 
o OO o o 
ooooo- 



000-0 


9 0 


i ! - i 
0 0 0 — 0 


1 • 
00 


000 — 0 


00 


000—0 


00 


0 0 9—0 


0 0 


6 6 6—6 


! I 
O O 


000 — 0 


OO 


000 — 0 


00 



-000 
-000 



id 



QQ 

id 



id 



X X X X X 


xxx 


x X x X X 


xxx 


x X X X X 


xxx 


x X x X x 


xxx 


X X X X X 


xxx 


® X X x X 


xxx 


® © xxx 


xxx 


®®® x x 


xxx 




X X < x x 


xxx 


X X X x x 


xxx 


X X X x X 


xxx 


<J x X x X 


xxx 


x <3 x x < 


xxx 


© X X x x 


xxx 


® ® x< x 


X < x 


®@®x x 


xxx 



000 — 
000 — 
000 - — 



2211691 



3/5 




NOIiilWSNVtil* 



on. 

CM 



LJ 

< 

LL 



id 



>- 




a 


a 




o 


OCT 


CM 


»— a 


i 


zo 




UJU 










cr 






LU 
LUM 














cr 






LU 




M 




t— 








< 








cr 




t 





o 



LU§< 

got. 



O 

£ll 
£cr 



LA. 



D 
CM 
CSJ 



>-cr 

Q. LU 

o a 
cr o 

LJ 

z: £ 

LU O 



jog *W 



cr 
o 

11 
II 



GO 



JQ 
CM 



>- 
LU CC 

cr tu 

LL 51 



csf 



cct= 

LUZ 

53 



2 

2TLJLJ 



CSJ 




C£LU 
LLZ 



m — 



CM — 



LA 




2211691 

4/5 





id 



or 





5/5 



2211691 



FIG. 6 



Y(1,1) 


Y(2,1) 


Y(1,2) 


Y(2,2) 






Y{1 ,5) 


Y(2,5) 



Y(5,1) 



Y(5,2) 



CO 



!Y(5,5) 



C(1/l) 


C(2,1) 




C(5,D 


C(1,2) 


C(2,2) 


C(5,2) 










C(1,5) 


C(2,5) 




C(5,5) 



Ad ,1) 


A(2,1) 




A(5,1) 


A(1,2) 


A(2,2) 




A(5,2) 



™< 1 > F(m,1)^ 
fe^T 



AO,5) 



A(2,5) 



F(1.n)„ 



o O— O I 

: : ; i 
6— o i 



A(5,5) 



I. 

r o~o-o"| 

O O— Oi 

i t 

o-oi 



f 



o o 

O O— O 

o 6-6 



F(5m,1) 



o o— o 
o o— o 

o 6—C 



F(1,5n) 



F(5m,5n) 



2211691 

1 

"A picture coding apparatus and interpolation circuit" 

The present invention relates to a picture signal 
processor in which a picture signal (either a still picture 
or a motion picture) is coded in order to transmit or file 
5 it at a high efficiency or in which a picture signal 

spatially sampled is converted into a picture of different 
resolution. 

Previous known apparatus for coding picture signals 
have been used for transmitting or filing the picture 
10 signals. 

Among coding systems employed in the picture signal 
coders, one which is very excellent in points of the coding 
efficiency and the picture quality, is a block coding system 
which adopts an orthogonal transform. 

15 This system utilizes the fact that, when a signal 

within a block has been transformed into two-dimensional 
frequency components, electric power concentrates near a 
component of low order (for example, a D.C. term 
corresponding to a mean value). As the methods of the 

20 transform, there are the DCT (Discrete Cosine Transform), 
etc. (refer to, for example, N. Ahmed, et al; "Discrete 
Cosine Transform", IEEE Trans. Comput., vol. C-23, 1974-1). 

Such a still picture filing apparatus or motion picture 
transmitting apparatus employing the orthogonal transform 

25 has had various problems which are referred to below. 

For example, in picture resolution conversion including 



manipulation process as expansion, compression or rotation 
has been generally performed in a picture- frame unit. 

More specifically, the impulse response of an 
interpolation filter corresponding to the picture conversion 
is found, and the convolution between the impulse response 
and an input picture signal is taken. Since, however, this 
processing method results in the form of a two-dimensional 
convolution, it involves a large number of operations and 
can only be satisfactorily achieved by means of a high- 
speed processing circuit which consists of a large number of 
multiplier units. 

Furthermore, it is difficult to share a frame memory 
for storing an original picture and a picture as the 
processed result thereof, and moreover a frame memory for 
the interpolation process needs to be separately added. 
This has the disadvantage that the size of the circuit for 
the processing method greatly increases. 

An interpolation circuit for use with a prior- art 
picture signal processor utilizes a method wherein a two- 
dimensional picture signal is multiplied by two-dimensional 
operators so as to accumulate the products (a method based 
on the so-called two-dimensional convolution). 

This interpolation method involves a small number of 
calculations and has been effective as a general 
interpolation method: 

(a) in the case where the two-dimensional operators 
are comparatively few in number, for example, 3 x 3, or 

(b) in the case where the relationship between the 



position of a pixel to be obtained by the interpolation and 
the positions of the pixels of the original picture is 
regular (for example, the case where the former pixel lies 
at the middle point of the latter pixels). 

The method, however, has resulted in an increase in the 
number of calculations and enlargement in the size of the 
circuit. 

(a) in the case where the two-dimensional operators 
are relatively large in number, for example, where a block 
average is smoothed and then displayed in the progressive 
coding of a picture, or 

(b) in the case where the pixel to be obtained by the 
interpolation and the pixels of the original picture are not 
in a simple positional relationship such as the middle 
point, as in the conversion between PAL and NTSC signals. 

It is an object of the present invention to provide a 
picture signal coder which realizes the picture resolution 
conversion and the conversion of image manipulation 
processing at a very high speed and without the addition of 
any special circuit. 

It is another object of the present invention to 
provide a picture signal interpolator which is capable of 
interpolation processing with a comparatively few number of 
calculations. 

In a first aspect of the present invention the 
conversion process of a picture signal is performed in block 
unit, not in frame unit, in order to raise the speed of the 
process. In addition to the orthogonal transform matrix 
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stated previously, the products between this orthogonal 
transform matrix and matrices which realize the picture 
resolution conversation, the image manipulation process such 
as expansion, compression and rotation, and various kinds of 
5 linear filtering are provided in a coefficient memory as new 
transform matrices, which are properly changed-over in 
accordance with the content of the conversion process in 
order to perform an orthogonal transform or an inverse 
orthogonal transform. 
10 In this manner, in the block coding apparatus which 

employs the orthogonal transform, the products between the 
inverse orthogonal matrix and the matrices for image 
manipulation and filtering processing are adopted as new 
inverse transform matrices, whereby the above processes can 
15 be realized very fast and easily. Further, in the 

communications of a motion picture, the values of orthogonal 
transform coefficients which are not transmitted are reduced 
by the use of the filtering process, whereby the elimination 
of a block distortion harmful in the picture quality and the 
20 enhancement of the transmission efficiency can be realized. 

In a second aspect of the present invention an original 
picture signal is once converted into two-dimensional 
frequency components by the use of the orthogonal transform 
matrix, whereupon they are converted into a spatial area 
25 again by the use of an interpolating matrix, whereby the 
amount of calculations of the interpolation process is 
reduced. Further, the products between the orthogonal 
transform matrix and the interpolating matrix are provided 



5 

in a transform constant memory as a new transform matrix 
beforehand, and they are properly changed-over in accordance 
with the position of a pixel to be interpolated or the kind 
of the interpolation, whereby a higher processing speed can 

5 be realized. 

In this manner, the picture signal is once converted 
into the two-dimensional spatial frequency components by the 
orthogonal transform, and they are subsequently converted 
into the spatial area again by the inverse orthogonal 
10 transform, whereby the interpolation process of a picture 
can be realized very flexibly and simply. Further, the 
products of the matrices for the orthogonal transform as 
well as the inverse orthogonal transform and any desired 
linear transform are calculated and written into the 
15 transform constant memory beforehand, and they are properly 
selected and used, whereby very fast and simple 
interpolating and filtering processes become possible. 
Further, in the communication of a picture employing the 
orthogonal transform, block averages are first transmitted, 
20 and differences from a curved surface obtained by 

interpolations from the averages are transmitted, whereby 
the transmitting period of time can be shortened. At this 
time the above differences are the differences of orthogonal 
transform coefficients themselves, and they can be obtained 
25 very simply without requiring any complicated process. 

The present invention will now be described in greater 
detail by way of examples with reference to the accompanying 
drawings, wherein: - 



Fig. 1 is a diagram showing the arrangement of a filing 
apparatus for still pictures which has image manipulation 
functions such as expansion, compression and rotation; 

Fig. 2 is a conceptual diagram showing the positional 
relationship between the frame of a picture and blocks in 
the case of performing an orthogonal transform; 

Fig. 3 is a diagram showing the arrangement of a motion 
picture transmitting apparatus; 

Figs. 4A to 4C are diagrams showing the relationship 
between the orthogonal transform coefficients and filtering 
process of a motion picture coding apparatus; 

Fig. 5 is a diagram showing an embodiment of a picture 
interpolation circuit according to the present invention; 
and 

Fig. 6 is a conceptual diagram showing the relationship 
between the D.C. values and interpolated pictures of a 
picture divided into blocks. 

Firstly, the process of converting a picture signal in 
a transmitting or recording mode will be explained. 

Referring to Fig. 1, a picture signal taken by a 
television camera 1 is spatially sampled and is thereafter 
separated by a Y/C separation circuit 2 into a luminance 
signal (Y) and colour-difference signals (CI, C2), which 
are respectively written into a frame memory 3. The 
contents of this frame memory are read out every block, 
e.g., 3a or 3b in which pixels 31 are put together as 
illustrated in Fig. 2, and they are converted by an 
orthogonal transform circuit 4 into coefficients which 
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correspond to frequency components. On this occasion, the 

values of a coefficient matrix (for example, DCT) to be used 

for the conversion are fetched from a coefficient memory 5. 

The coefficients after the conversion are quantized by a 
5 quantizer 6, and the quantized coefficients are transferred 

to an entropy coder 7 and are coded into signals with 

redundancy removed, which are recorded in a video file 8. 

The content of the conversion which is performed by the 

orthogonal transform circuit 4, will now be described. 
10 When the divided parts of the picture signal are 

expressed by, for example, a matrix of 8 x 8 elements; D(8, 

8), the following formula holds: 
C = T . D . T" 1 

wherein T (T* 1 ) denotes an orthogonal transform matrix, 
15 which becomes the following equation (1) in the case of the 

DCT: 

T ±J = k [K, . cos {i(j + h) */8}] (i, 3 = 0, 7) 

...(1) 

where K ± = 1/ Si for i + 0, and 

20 K. = 1 fori £ 0. 

As the result of the conversion, the coefficient matrix 

C(i, j) corresponding to the frequency components within the 

block is generated. In this case C(0, 0) is a D.C. 

coefficient corresponding to the average value of the block, 
25 and the coefficients are of higher spatial frequencies as i 

and 2. increase. 

In general pictures, significant coefficients 

concentrate on coefficients of comparatively low orders, so 
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that the removal of redundancy is permitted by the 
assignments of quantization and variable- length coding 
utilizing this property. 

On the other hand, in a reproducing mode, the variable- 
5 length codes read out of the video file 8 are decoded by an 
entropy decoder 9 into the orthogonal transform 
coefficients, which are passed via an inverse quantizer 10 
and then inverse-transformed into the original picture 
signal every block by an inverse orthogonal transform 

10 circuit 11. On this occasion, a coefficient matrix to be 

used for the inverse transform is fetched from a coefficient 
memory 12. The reproduced picture signal is passed via a 
frame memory 13 and has a luminance signal and colour- 
difference signals composed by an encoder 14, whereupon the 

15 resulting picture is displayed on a monitor television set 
15. 

The above circuit arrangement is controlled by a 
controller 16 in accordance with the contents of commands 
given from a keyboard 17 . 
20 The content of the conversion which is performed by the 

inverse orthogonal transform circuit 11 will now be 
described. 

In an ordinary case where any image manipulation 
function is not used, the picture within the block is 
25 subjected to the following transform: 

D = T- 1 . C . T ...(2) 
wherein the values of the DCT matrices T and T" 1 are 
fetched from the coefficient memory 12. 



On the other hand, in the case where the picture within 
the block is processed by any image manipulation function, 
the following transform is carried out: 

D = (L . T" 1 ) . C . (T . R) • . .(3) 

5 wherein L and R denote matrices each having 8x8 elements, 
and the products between each of them and the corresponding 
one of T and T" 1 become a matrix in the same form. Also the 
values of the two transform matrices (L . T" 1 ) and (T . H) 
are fetched from the coefficient memory 12 in accordance 
10 with signals from the controller 16, and the processes of 
the image manipulation and the inverse transform are 
simultaneously performed by the inverse orthogonal transform 
circuit 11 • 

The contents of the matrices L and R can be expressed 
15 % depending upon the kinds of the image manipulation, as 
indicated below by way of examples: 

(1) The case of Horizontal Inversion: 
L = E 0, R = E 90 
where E 0; unit matrix, and 
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(2) The case of Vertical Inversion: 
L = E 90, R = E 0 

(3) The case of 90° Right Turn: 

C — ^ fc C (transposition) , and 
L = E 0, R = E 90 

(4) The case of 90° Left Turn: 

C — ^ t C ( transposition ) , and 
L = E 90, R - E 0 

(5) The case of 180° Rotation: 
L = E 90, R = E 90 

(6) The case of Expansion (in Horizontal Direction) 
L = E 0, R = MH 

For doubling a left half, 
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(7) The case of Expansion (in Vertical Direction): 
L = MV, R = E 0 

For doubling an upper half, 



MV= 
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(8) The case of Compression: 

When an expansion matrix consisting of the 
coefficient matrices C 11, C 12, C 21 and C 22 of four 
adjacent blocks: 



C = 



is compressed to h, 



C 11 
C 21 



C 12 
C 22 



D 11 
D 21 



D 12 
D 22 



T -i 
0 



0 
T -i 



C 11 
C 21 



C 12 
C 22 



where the calculations of even-numbered (odd-numbered) 
and columns are not executed. 

(9) The case of Low Pass Filtering (LPF): 
L = KL , R = KR 

where 
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KL = 



KR = 
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In this case, any desired low-pass characteristics can be 
bestowed by changing the values a and b (subject 
to 0 ^ a, b ^ 0.5). 

Although, in the above, the various examples of the 
image manipulation and the filtering have been described, it 
is to be understood that similar effects are attained also 
for processes of still higher degree and complicated 
combinations thereof. 

Although in the above examples, the matrices L and R 
have been assumed to consist of 8 x 8 elements which have 
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the same form as that of the picture D or the transform 
coefficients C, it is possible, however, that the picture D 
to be extracted is expanded into, for example, 12 x 12 
elements on the sending side, while the matrices L and R are 
respectively set at 8 x 12 elements and 12 x 8 elements. On 
this occasion, the following formula 
(4) holds: 

C^T.L.D.R.T" 1 .-.(4) 
In this manner, regarding any desired linear transform 
which is expressed by a matrix, the products between the 
matrix and an inverse orthogonal transform matrix are read 
out of a memory circuit, and they are multiplied from both 
the sides of the coefficient matrix, whereby the conversion 
of a picture and the inverse orthogonal transform can be 
simultaneously performed, and the operating period of time 
can be reduced to a half or less. 

Circuits required for the processing stated above are 
only the memory circuit for storing the matrices as 
mentioned above and some control circuits for changing-over 
the matrices in accordance with the kinds of the transforms, 
and they can be realized very simply. In addition, a system 
which rewrites the contents of the memory circuit adaptively 
by the use of a microprocessor can be readily realized. 

Next, a second embodiment of the present invention will 
be described with reference to Fig. 3 and Figs 4A - 4C in 
connection with a motion-compensated interframe coding 
equipment of low bit rate. 

As in the first embodiment, the output of a television 
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camera 1 is separated by a Y/C separation circuit 2 into 
luminance and colour-difference signals, which are 
respectively written into a frame memory 3. The picture 
signal is divided into blocks each having 8x8 pixels, and 
the difference thereof from the signal of a transmitted 
frame is input to an orthogonal transform circuit 4 by a 
subtracter 18. On this occasion, the motion vector between 
the frames is measured so that the difference may be reduced 
(motion-compensated interframe prediction). 

In the orthogonal transform circuit 4, the products 
between a transform matrix and a matrix having low-pass 
characteristics are read out of a motion estimator or 
coefficient memory 5 as stated in the first embodiment. The 
low-pass characteristics are adaptively controlled in 
accordance with the number of transmission frames. The 
coefficients after the conversion are passed through a 
quantizer 19 and have redundancy removed by an entropy coder 
7, and the resulting codes are delivered to a transmission 
line 24 via a line interface 23a. 

In addition, the output of the quantizer 19 is coded 
into an interframe difference signal by an inverse quantizer 
20a as well as an inverse orthogonal transform circuit 11a, 
the difference signal is added with a motion vector- 
compensated transmitted frame signal by an adder 22a, and 
the resulting signal is written into a frame memory 6 as a 
new frame signal. By means of a motion compensation circuit 
21a, the output of the frame memory 6 becomes a reference 
picture which is compared with a frame signal to be 



subsequently transmitted. 

On the other hand, the signal of the transmission line 
24 is decoded into an orthogonal transform coefficient by a 
line interface 23b as well as an entropy decoder 9, and the 
coefficient is transformed into an interframe difference 
signal by an inverse quantizer 20b as well as an inverse 
orthogonal transform circuit lib. As in the case of the 
sending side, this interframe difference signal is added 
with a motion vector-compensated transmitted frame signal by 
an added 22b, and the resulting signal is written into frame 
memories 10 and 13 as a new frame signal. This signal is 
passed through a Y/C encoder 14, and is displayed on a 
monitor TV set 15. 

On this occasion, the various circuits including a 
motion compensation circuit 21b are controlled so that the 
contents of the frame memory 10 may agree with those of the 
frame memory 6 on the sending side. 

Incidentally, in the case of an inverse orthogonal 
transform, both the sending and receiving sides employ an 
ordinary inverse transform matrix in consideration of the 
compatibility (but it is also possible to execute various 
kinds of image manipulation processes and filtering 
processes described in the first embodiment ) . 

Next, a process for eliminating a block distortion will 
be described with reference to Figs. 4A to 4C. 

In general, when all coefficients subjected to the 
orthogonal transform are transmitted, the block distortion 
hardly poses a problem. It is sometimes the case, however, 
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that significant coefficients of comparatively high orders 
fail to be transmitted due to the security of the number of 
transmission frames. By way of example, when pixel signals 
shown in Fig. 4A are subjected to the orthogonal transform, 
the transformed coefficients become as shown in Fig. 4B. 
Here, marks (§) denote significant coefficients which are 
transmitted, marks x denote insignificant coefficients which 
are not transmitted, and marks A denote coefficients which 
are significant but which cannot be transmitted. On the 
receiving side, the values of the untransmitted coefficients 
are decoded as zero at this time, so that a level difference 
arises between blocks under the influence of the 
coefficients indicated by the marks A and appears as an 
obstacle in the picture quality. 

In the present invention, in order to prevent such 
degradations in the picture quality, when the number of 
transmission frames is secured, the orthogonal transform is 
performed in the way that the products between the 
orthogonal transform matrix and the matrix having the low- 
pass characteristics are fetched from the coefficient 
memory 5. Since the comparatively high frequency components 
of the picture are removed by the low-pass filtering, the 
values of the untransmitted coefficients become small as 
illustrated in Fig. 4C. 

As a result, the picture quality degradations 
ascribable to the block distortion can be eliminated or 
relieved very effectively. It is also possible to control 
the number of transmission frames in such a way that the 



filtering characteristics are adaptively controlled in 
accordance with the picture to-be-transmitted. 

Since the above filtering process is performed 
simultaneously with the orthogonal transform, it is very 
5 fast. It can be readily realized merely by increasing the 
area of the coefficient memory. 

Moreover, coefficients which are hardly problematic in 
the picture quality even when not transmitted are uniquely 
determined by the filtering characteristics • By coding the 
10 characteristics and transmitting the codes, therefore, the 

transmission of the invalid coefficients can be omitted, and 
the enhancement of the transmission efficiency can be 
achieved. 

In this embodiment, the multiplier- accumulator circuits 
15 are employed for the operations • However, in a case where 
the processing period of time has some margin, signal 
processors can also be utilized. 
[Embodiment 3] 

The third embodiment of the present invention will how 

20 be described with reference to Fig. 5. 

The circuit arrangement shown in Fig. 5 converts an 
original picture of m x n pixels into an interpolated 
picture of M x N pixels. A digital picture signal which has 
been accepted from a television camera 51 and an A/D 

25 converter 52, from a still picture file 53 or from a still 
picture transmission equipment 54 is once written into a 
frame memory 55. Signals of m x n pixels; V(i, j) (i = 1, 
2, . . . m) ( j = 1, 2, ... n) are read out of the frame memory 
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55, and they are subjected by a multiplier-accumulator 
circuit 56 to multiplying accumulation operations with 
signals read out of a coefficient or transform constant 
memory 58, thereby to be transformed into coefficients 
corresponding to frequency components; C(i, j) 
(i = 1, 2, ... m) (j = 1, 2, ... n). This transform is a 
linear transform, and is expressed as follows by the use of 
transform matrices Dl(i, j) (i, j - 1, 2, . . . n) and 
D2(i, j) (i, j = 1, 2, ... m): 

C = Dl . Y . D2 ... (5) 

Although transform matrices include those of KL 
transform, Hadamard transform, etc., DCT (Discrete Cosine 
Transform) of comparatively easy operations and excellent 
characteristics is used. In case of the DCT, the matrices 
Dl and D2 become as follows: 

Dl(i, j) =>/27n [Kj.cosUi - h) (j - l)*/n}] 

(i, j - 1, 2, ... n) ...(6) 
D2(j, i) =\/27ra [K j .cos{(i - h) (j - D ft/m}] 
(i, j = 1, 2, ...m) ...(7) 
where = 1//2" for j = 1, and 

K } = 1 for j i 1 

The matrices Dl and D2 are normalized orthogonal 
transform matrices (unitary matrices), and the products 
thereof form a unit matrix at m = n. 

The coefficients transformed by the multiplier- 
accumulator circuit 56; C(i, j) (i = 1, 2, ... m) 
(j = 1, 2, ... n) are components which correspond to a kind 
of spatial frequencies. C(l, 1), for example, is a D.C. 
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coefficient corresponding to the average value of a block, 
and the coefficients are of higher spatial frequencies as i 
and j_ increase. 

These coefficients are once stored in a buffer memory 

60. 

Subsequently, matrices for an interpolation process are 
read out of the coefficient memory 58, and the 
multiplications thereof with the above transform 
coefficients are executed by the multiplier-accumulator 
circuit 56, whereby an interpolated picture F is generated. 
This interpolated picture F is written into a frame memory 
59, and is displayed on a monitor (display unit) 62 via a 
D.A converter 61. These circuits are controlled by a 
controller 57. 

The following formula (8) holds for the transform 
stated above: 

F = HI . C . H2 .-*(8) 
In addition, HI and H2 become as follows: 

Hl(j, i) =s/27n [K. .cos{(i - h) (j - 1) 7T/N)] 

(i = 1, 2, ... N) (j = 1, 2, ...n) ...(9) 
H2(i, j) =y57M [K j .cos{(i - h) (j - l)ft/M}] 

(i = 1, 2, ... N) (j = 1, 2, ... m) ...(10) 
where Kj = 1//2 for j = 1, and 
K. = 1 f or j / 1 

Owing to the above operations, the original picture 
of m x n pixels is converted into the interpolated picture 
of M x N pixels by the DCT matrices Dl, D2 and the 
interpolation matrices HI, H2. 



Although, in the above, the process for obtaining the 
orthogonal transform coefficients and the process for 
generating the interpolated picture on the basis of these 
coefficients have been described as two separate stages, 
5 they are realized by a single process in this embodiment. A 
detailed explanation of the single process will now be 
given. 

When equation (5) is substituted into equation (6), the 
following formula is obtained: 
10 F = HI . Dl . Y . D2 H2 

= (HI Dl) . Y . (D2 H2) ...(11) 
wherein (HI . Dl) and (D2 . H2) denote matrices of (n x N) 
and (M x m), respectively, and they can be uniquely obtained 
beforehand irrespective of the contents of the original 
15 picture when the positional relations between the original 
picture and pixels to be interpolated are determined. 

Accordingly, the contents of these matrices are 
previously written into the coefficient memory 58, whereby 
the interpolation process can be realized by very fast and 
20 simple operations. 

It should be noted that in the case where no 
interpolation is performed (M = m, N = n), the matrices 
(HI . Dl) and (D2 . H2) become the unit of (n x n) and 
(m x m), respectively, and the original picture Y is 
25 reproduced as it is. 

There will be explained the relationship between the 
orthogonal transform coefficients and the picture in the 
above interpolation process* 



When the transform coefficients of equation (5) are 
brought into correspondence with original picture areas, 
they are expressed as cosine curved surfaces corresponding 
to the orders thereof- For example, the coefficient C(2, 1) 
5 becomes a curved surface which varies in a horizontal 
direction at a half of the cosine cycle. This curved 
surface is continuous, and the values of any desired points 
lying thereon are significant for the interpolated picture. 

Accordingly, although i and j_ have been defined as 
10 integers in equation (9), they need not always be the 

integers. In the case where interpolation points are at 
unequal intervals, any desired real numbers can also be 
employed. 

Further, besides the interpolation process stated 
15 above, the filing process etc. of the picture can be 
simultaneously realized. 

B Y wa Y of example, any desired matrices W and W f are 
multiplied from both the sides of the original picture Y or 
the interpolation picture F, whereby low-pass and other 
20 filtering processes can be realized. More specifically, 

instead of the matrices (HI . Dl ) and (D2 . H2) in equation 
(11), matrices (W . HI . Dl) and (D2 . H2 . W), 
(HI . Dl . W) and (W . D2 . H2), or (HI . W . Dl) and 
(D2 . W . H2) are written into the coefficient memory 58 
25 beforehand, and the multiplications thereof with the 

original picture Y are executed, whereby the speed of the 
operations can be readily raised. 

In addition, transform constants are sometimes 
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arranged very regularly in these matrices, and a high-speed 
operating method called " FFT " is also applicable. 

Although, in the above, the several examples of the 
interpolation and filtering of pictures have been described, 
it is to be understood that similar effects are attained 
also for processes of still higher degree and complicated 
combinations thereof. 

Circuits required for such processes are only the 
memory circuit for storing the matrices stated above and 
some control circuits for changing-over the matrices in 
accordance with the kinds of the transforms, and they can be 
realized very simply. 

Further, it has been described above the picture 
signals accepted from the television camera, the still 
picture file and the still picture transmission equipment 
are handled. In this regard, in the transmission equipment 
for the still picture or a transmission equipment for a 
motion picture, the picture is divided into blocks, the 
transform such as DCT is performed every block, and the 
transformed coefficients are entropy-coded and then 
transmitted. Accordingly, in the case where the 
coefficients are inverse-transformed on the receiving side, 
circuits therefor and arithmetic circuits indicated by 
equation (8) of this embodiment can also be shared. 

Besides, the interpolation process described in this 
embodiment can also be realized using digital signals 
processors which have multiplying accumulation operation 
functions of very high speed. 
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Further, using these processors, a system in which the 
contents of the memory circuit are adaptively rewritten can 
be realized with ease. 

In this manner, regarding any desired linear transform 
5 which is expressed by a matrix, the products between the 
matrix and an inverse orthogonal transform matrix are read 
out of a memory circuit, and they are multiplied from both 
the sides of the coefficient matrix, whereby the conversion 
of a picture and the inverse orthogonal transform can be 
10 simultaneously performed, and the operating period of time 
can be shortened to a half or less. 

The fourth embodiment of the present invention will now 
be described with reference to Fig. 6. 

The third embodiment has been described as to the 
15 method of obtaining the interpolated picture of the M x N 
pixels interpolated from the original picture of the m x n 
pixels. However, when the interpolation process is 
performed using the average of, for example m x n blocks 
left intact, the average values of the individual blocks are 
20 not always ensured. 

In this embodiment, therefore, an interpolation method 
in which the D.C. values (average values) of a picture 
divided into blocks are kept will now be described. 

An example of interpolation which employs the average 
25 values of a block to be subjected to an interpolation 

process and the surrounding 24 blocks is illustrated in 
Fig. 6. 

Now, when an original picture Y(5, 5) consisting of 5 x 
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5 blocks is subjected to the orthogonal transform by the 
method DCT stated in the third embodiment, 5x5 transform 
coefficients C(5, 5) can be found, and an interpolated 
picture F(M, N) can be obtained from the coefficients by the 
use of equation ( 8 ) . 

Assuming here that each of the blocks is configured of 
m x n pixels, the following holds: 
F = HI . c . H2 

M = 1, 2, ... 5m, N = 1, 2, ... 5n 

where HI and H2 denote the respective matrices stipulated by 

equation (9) and equation( 10 ) - 

That is, the interpolated picture F(M, N) can be 

expressed using the 25 unknown transform variables 

■C(i, j) (i = lr 2, ... 5) (j = 1, 2, ... 5). 

Accordingly, letting A(i, j) (i = 1, 2, — 5) 

(j = 1, 2, ... 5) denote the averages of the respective 

blocks, the following formula holds: 

A(i, j) = H F(i + k, j + 1) ...(12) 
k,l 

Thus, 25 linear equations whose number is equal to that 
of the unknown variables are generated, and the 
aforementioned transform coefficients C(i, j) can be 
evaluated by solving the simultaneous equations. 

That is, the following equations hold: 

a = W . c . . .(13) 

c = W . a ... (14) 

where c and a denote vectors of 25 terms into which the 
respective matrices C and A of 5x5 elements are rearrayed 
to be undimensional . Since the matrix W does not depend 
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upon the original picture, it is computed beforehand and is 
written into the transform constant memory 58, whereby the 
interpolation process can be realized using a method similar 
to that of the third embodiment. 

That is, the matrix C is determined from the vector c, 
and the interpolated picture F(M, N) in which the block 
averages are kept can be obtained using equation ( 8 ) . 

Although, in the above description, the original 
picture Y(5, 5) composed of the 5x5 blocks has been 
referred to, it is to be understood that the original 
picture can be expanded to any desired number of blocks. 

Furthermore, as in the third embodiment, any of the 
low-pass and other filtering processes can be simultaneously 
realized. 

Moreover, regarding the motion picture or still picture 
transmission equipment wherein the orthogonal transform such 
as DCT is performed every block, differences from the 
interpolation pixels F(M, N) as evaluated from the D.C. 
values are transmitted, whereby the transmission period of 
time can be shortened. The reason is that, in the periphery 
of the block, deviations from the average value become 
large. More specifically, when the parts to be transmitted 
in the interpolated picture: 

F'(K, L) (K = 1, 2, ... m) (L = 1, 2, ... n) 
are extracted and are subjected to the orthogonal transform 
by the use of equation (5), the following formula holds: 

C = Dl . F . F' . D2 

= (Dl . HI) . C . (H2 . D2) ...(15) 
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wherein differences from interpolated pixels F' (M, N) as 
obtained from the D.C values are expressed as differences 
also in the area of orthogonal transform coefficients C f . 
Accordingly, the method of transmitting the differences 
between the coefficients C f and the coefficients obtained by 
the orthogonal transforms of the blocks of the original 
picture is effective for enhancing the transmission 
efficiency. In addition, as in the foregoing, it is 
possible to apply the method wherein the values of the 
matrices (Dl . HI) and (H2 . D2) are computed beforehand, 
and they are fetched from the transform constant memory 58. 
Incidentally, as to the aforementioned transmission 
efficiency, an improvement of about 8% has been expected by 
a simulation. 

Processes conforming to the contents of the above 
interpolation and filtering processes can be realized 
through quite similar controls merely by increasing the area 
of the transform constant memory, so that the method of the 
embodiment is very flexible. 

In this embodiment, the multiplier- accumulator circuits 
are employed for the operations. However, in the case 
where the processing period of time has some margin, signal 
processors can also be utilized. 
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CLAIMS : 

1 ♦ A picture coding apparatus in which a picture signal is 
divided into blocks, a difference signal between the picture 
signal or a picture signal belonging to the block and a 
transmitted picture signal is multiplied by a first 
orthogonal transform matrix so as to be converted into 
coefficients, and the coefficients have redundancy removed 
using a variable-length code or the like; wherein in 
addition to the first orthogonal transform matrix, products 
between said first orthogonal transform matrix and desired 
matrices are comprised as a group of second transform 
matrices, and wherein an orthogonal transform or an inverse 
transform is performed by properly changing-over said group 
of second transform matrices, whereupon the transformed 
signal is transmitted, recorded or reproduced. 

2. A picture coding apparatus according to claim 1, 
wherein products between said first orthogonal transform 
matrix and products of matrices for picture resolution 
conversion and image manipulation processes such as 
expansion, compression and rotation of picture signals, 
matrices for filtering processes such as low-pass and high- 
pass filtering processes, or/and matrices of combinations 
thereof are comprised as said group of second transform 
matrices - 

3. A picture coding apparatus according to claim 1, 
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wherein the picture signal or the difference signal is 
properly subjected to any one of a number of filtering 
processes having different characteristics so that, among 
the coefficients linearly transformed by the use of any of 
said second group to transform matrices, ones not to be 
transmitted may become smaller in absolute values than ones 
to be transmitted, whereupon the processed signal is 
transmitted, recorded or reproduced. 

4. A picture interpolation circuit comprising means for 
dividing a picture signal into blocks, and for multiplying 
matrix-shaped pixel signals belonging to said each block, by 
each of a first orthogonal transform matrix and a second 
transposed orthogonal transform matrix from both sides 
thereof, so as to be transformed into coefficients, and 
means for multiplying the transform coefficients by products 
between a third inverse transposed orthogonal transform 
matrix and a fourth inverse orthogonal transform matrix or 
between said third matrix and any desired matrix, and 
products between said fourth matrix and any desired matrix 
from right and left sides thereof, respectively. 

5. A picture interpolation circuit according to claim 4, 
further comprising means for multiplying said pixel signals 
from the left side thereof by products among said first 
orthogonal transform matrix, said fourth inverse orthogonal 
transform matrix and any desired matrix and means for 
multiplying said pixel signals from the right side thereof 



29 

by products among said second transposed orthogonal 
transform matrix, said third inverse transposed orthogonal 
transform matrix and any desired matrix. 

6. A picture coding apparatus constructed substantially as 
5 herein described with reference to and as illustrated in 

Figs. 1 to 4 of the accompanying drawings, 

7. A picture interpolation circuit constructed 
substantially as herein described with reference to and as 
illustrated in Figs. 5 and 6 of the accompanying drawings* 
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